fix(session): scrub TMUX vars + PTY-exit circuit breaker (COD-115/COD-118)#147
Open
aakhter wants to merge 3 commits into
Open
fix(session): scrub TMUX vars + PTY-exit circuit breaker (COD-115/COD-118)#147aakhter wants to merge 3 commits into
aakhter wants to merge 3 commits into
Conversation
…-loop
When the web server is launched from inside a tmux pane it inherits TMUX/
TMUX_PANE. tmux's nesting guard then makes every new attach-bridge PTY
(`tmux attach-session`, used by codex/opencode/gemini and mux-wrapped claude)
exit code 1; the respawn controller recreates the dead bridge → infinite loop.
The existing guard in buildMuxAttachEnv() used `TMUX: undefined` on a
{...process.env} spread, which leaves the KEY present with value undefined —
node-pty serializes it as the literal string "TMUX=undefined", still tripping
the guard. (The working create path in tmux-manager.ts uses `delete`.)
Fix:
- Primary: delete process.env.TMUX / TMUX_PANE at web bootstrap (src/index.ts)
so every downstream {...process.env} spread is clean regardless of launch
context. `delete`, not `= undefined`.
- buildMuxAttachEnv(): build a copy and `delete` TMUX/TMUX_PANE/CLAUDECODE
(and COLORTERM when not truecolor) instead of `: undefined` — same node-pty
quirk affected all of them.
- Test: assert the keys are genuinely ABSENT (`'TMUX' in env === false`), not
merely undefined — the prior test only checked `toBeUndefined()`, which is
why the bug slipped through. Red→green confirmed.
Verified on isolated beta launched from inside tmux (inherited the poisonous
TMUX=codeman,980,7): created a codex session + triggered interactive attach —
the bridge `tmux -L codeman-beta attach-session` spawned with NO TMUX in its
env, attached successfully, zero "exited with code: 1", server healthy.
Circuit-breaker for repeated non-zero bridge exits (AC bullet 4, optional)
split to a follow-up. Deploy-pending (substrate): never auto-deployed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…PTY exits Defense-in-depth after COD-115. If the interactive PTY exits non-zero repeatedly within a short window, recovery/reconnect paths recreate it indefinitely (COD-115 saw 114 'exited with code: 1' events + orphans). - New pure InteractivePtyExitBreaker (session-pty-exit-breaker.ts): injectable time, sliding window, clean-exit resets counter, stays tripped until reset(). Defaults: threshold 5, window 10s. - Session records each interactive PTY exit in the breaker; on trip it flips _status to 'error', sets _respawnBlocked, emits respawnBreakerTripped. startInteractive() refuses to respawn while blocked, so all recovery/reconnect callers stop looping uniformly. - Explicit user restart (POST /api/sessions/:id/interactive) calls resetRespawnBreaker() so intentional restarts are never blocked. - New SSE event session:respawnBreakerTripped wired in sse-events.ts + constants.js (registries in sync) + session-listener-wiring.ts; minimal diagnostic toast in app.js. - Tests: test/respawn-pty-breaker.test.ts (pure trip/reset/window + MockSession session-level trip/reset). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
POST /api/sessions/:id/interactive calls session.resetRespawnBreaker() before startInteractive(); mock missing the stub → route threw → 422.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related fixes for tmux-backed session stability:
COD-115 — Delete TMUX/TMUX_PANE (don't set-to-undefined)
When the Codeman server is launched from inside a tmux session,
process.env.TMUXis set. The mux-attach PTY inherits that env, and tmux detects the nesting viaTMUX/TMUX_PANE→ exits 1 → Codeman's respawn kicks in → crash-loop.Root cause:
buildMuxAttachEnv()was spreading{...process.env, TMUX: undefined}. node-pty serializes that as the literal string"TMUX=undefined"(the key is present with a non-empty value), so the guard still fires. Fix:delete env.TMUXanddelete env.TMUX_PANE— key absent from the environment entirely. Same fix applied at web bootstrap insrc/index.tsfor belt-and-suspenders coverage.COD-118 — PTY-exit circuit breaker
Introduces
InteractivePtyExitBreaker(pure class, injectable time for tests): a sliding-window breaker that trips after 5 non-zero PTY exits within 10 seconds, sets_respawnBlocked, and emitsrespawnBreakerTripped. SSE event + push notification tell the user. Explicit user restart (POST /api/sessions/:id/start) resets the breaker before relaunching.Prevents a misconfigured case (bad shell, bad Claude install, permission denied) from hammering tmux/PTY in an infinite respawn loop and burning system resources.
Files changed
src/index.ts— delete TMUX/TMUX_PANE at web bootstrapsrc/session-cli-builder.ts—buildMuxAttachEnv()usesdeletefor TMUX, TMUX_PANE, CLAUDECODE, COLORTERMtest/session-cli-builder.test.ts— new; COD-115 absence assertions ('TMUX' in env === false)src/session-pty-exit-breaker.ts— new;InteractivePtyExitBreaker(sliding window, threshold 5/10s)src/session.ts—_ptyExitBreaker+_respawnBlocked,resetRespawnBreaker(), PTY exit instrumentationsrc/web/session-listener-wiring.ts—respawnBreakerTrippedhandler → SSE + push + run-summarysrc/web/sse-events.ts—SessionRespawnBreakerTrippedeventsrc/web/public/constants.js—SESSION_RESPAWN_BREAKER_TRIPPEDconstantsrc/web/public/app.js— diagnostic toast onrespawnBreakerTrippedsrc/web/routes/session-routes.ts—session.resetRespawnBreaker()beforestartInteractive()test/respawn-pty-breaker.test.ts— new; 13 tests (pure breaker + session integration)Test plan
npm test -- test/respawn-pty-breaker.test.ts test/session-cli-builder.test.ts→ 15/15 passtsc --noEmit→ cleannpm run build→ clean